{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ad6b04d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import yfinance as yf\n",
    "# Download Apple Ticker Data\n",
    "data = yf.download('AAPL', start='2009-04-01', end='2010-01-01', interval='1d')\n",
    "print(len(data.index))\n",
    "for i in range(4, len(data.index), int(len(data.index)/10)):\n",
    "    data.iloc[i] = np.nan\n",
    "data.to_csv('AAPL_2009-2010_6m_RAW_1d.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4da38ad",
   "metadata": {},
   "source": [
    "# 1. Libraries & Sample Data\n",
    "The first step is to load our Python Libraries and download the sample data. The dataset represents Apple stock price (1d bars) for the year 2010"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1aa05430",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load Python Libraries\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "from IPython.display import display, HTML\n",
    "# for dataframe display\n",
    "pd.set_option(\"display.max_rows\", None)\n",
    "def display_df(df):\n",
    "    # Puts the scrollbar next to the DataFrame\n",
    "    display(HTML(\"<div style='height: 200px; overflow: auto; width: fit-content'>\" + df.to_html() + \"</div>\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fa831031",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download Sample Data\n",
    "data = pd.read_csv('AAPL_2009-2010_6m_RAW_1d.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c4a4a35",
   "metadata": {},
   "source": [
    "# 2. Exploratory Data Analysis\n",
    "Next, we want to analyze our data. Display the data as a dataframe, and plot some relevant data so you can get an idea of what our dataset looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aedbfa86",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display as Dataframe\n",
    "display_df(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b0f9772",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Index data by Date\n",
    "data.set_index('Date', inplace=True)\n",
    "display_df(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b401f82f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the Close Data\n",
    "data['Close'].plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "189bc851",
   "metadata": {},
   "source": [
    "# 3. Data Cleaning\n",
    "Next, we need to clean our data for training our model. This requires removal of NaN values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b0ebfafd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check for null values\n",
    "print('Number of Null Values =', data.isnull().sum())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d9128668",
   "metadata": {},
   "outputs": [],
   "source": [
    "# forward fill missing values\n",
    "data=data.ffill()\n",
    "display_df(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a6050c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check for null values\n",
    "print('Number of Null Values =', data.isnull().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "764fed7f",
   "metadata": {},
   "source": [
    "# 4. Feature Definition\n",
    "Now that we have cleaned our stock data, we can define some financial indicaors, or \"features\" to train our model on. We will be calculating some popular indicators: 20-day Close Moving Average, 5-day Close Moving Average, 20-day Close Bollinger Bands, and 20-day Historical Volatility of Close Price. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dcb99ad6",
   "metadata": {},
   "outputs": [],
   "source": [
    "data['MA5'] = data['Close'].rolling(window=5).mean()\n",
    "data['MA20'] = data['Close'].rolling(window=20).mean()\n",
    "data['STD20'] = data['Close'].rolling(window=20).std()\n",
    "data['BB_upper'] = data['MA20'] + (data['STD20'] * 2)\n",
    "data['BB_lower'] = data['MA20'] - (data['STD20'] * 2)\n",
    "data['Log_Ret'] = np.log(data['Close'] / data['Close'].shift(1))\n",
    "data['Vol20'] = data['Log_Ret'].rolling(window=20).std() * np.sqrt(252)\n",
    "display_df(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42224d3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove rows with MA=NaN\n",
    "data = data.dropna(axis=0)\n",
    "display_df(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e8f48c1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot Features: Close, MA20, BB Upper, BB Lower\n",
    "data['Close'].plot()\n",
    "data['MA20'].plot()\n",
    "data['BB_upper'].plot()\n",
    "data['BB_lower'].plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e01e1670",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot Features: Close, MA20, MA5\n",
    "data['Close'].plot()\n",
    "data['MA20'].plot()\n",
    "data['MA5'].plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bfbddc5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot Features: Volatility\n",
    "data['Vol20'].plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6862f7cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "data.to_csv('AAPL_2009-2010_6m_all_features_1d.csv')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "rl-project-udacity",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
